Self-Organizing Maps of Words for Natural Language Processing Applications

Kohonen’s Self-Organizing Map (SOM) is a general unsupervised tool for ordering highdimensional statistical data so that neighboring nodes on the map represent similar inputs. Often the SOM is applied to numerical data in application areas like pattern recognition, signal processing, and multivariate statistical analysis. The SOM can also be used to find statistical similarities between symbols if suitable contextual information is available as there indeed is for words in natural language texts. Using the short contexts of each word occurring in a text the SOM algorithm is able to organize the words into grammatical and semantic categories represented on a two-dimensional array. The similarity of the categories is reflected in their distance relationships on the array. This kind of a word category map may then be utilized in applications such as the analysis of large document collections. In addition to the description of information retrieval and textual data mining applications, this paper outlines the relation of word category maps to the symbolic knowledge representation formalisms. The graded nature of the categorization performed by the SOM is discussed. The aim is also to provide an overview on the research results and on the potential new areas.